Many data storage media, such as data storage optical disks, have a so-called fixed block architecture (FBA) format. Such format is characterized in an optical disk by so-called hard sectoring the disk's single spiral track into a plurality of sectors. Every one of the sectors have identical data storage capacity, i.e. 512 bytes, 1024 bytes, 4096 bytes, etc. Because of the FBA disks and the variability of data lengths of compressed data with respect to the source uncompressed data, in-line data compression has not been employed with FBA formatted disks. It is desired to efficiently store and enable simple random address accessing a variable amount of compressed data resulting from a compressing data formatted into addressable blocks. Such compressed data are then recorded on a FBA formatted disk. If the sector data does not compress to a fewer bytes, then the data are stored without data compression on the data storage disk.
It is also desired to maintain host processor addressability of the compressed data blocks within each compressed group of data blocks. It is also desired when compressing data for storage on a FBA storage medium to maintain a maximal addressability of all unused data storing sectors even though the number of sectors required to store the compressed data blocks is unknown. A further desire is to provide for random addressing of the compressed data blocks recorded in an FBA formatted storage medium.
The data pattern randomness of most input data streams and the variability in the resulting length of the compressed data output after the application of the various compression algorithms, does not allow for the prediction of the amount of storage space required to contain the compressed data. This situation requires a link between the transmission of the data stream to be compressed and recorded and the results of the compression process to assist the host processor in its storage management process.
The function of updating a data file in this environment can not use any usual data updating process (read, update, write back) because the data pattern as a result of the update may not compress to the same degree as the original data block and therefore updated compressed data most probably will not fit in the original storage space required to store the original data.
In an fixed block architecture (FBA) environment, data are recorded on a data storage medium in fixed sized units of storage called sectors where each recording track on the medium contains a fixed number of such sectors. The addressing convention for optical disk devices consists of a track address on the medium and a sector number of the particular track. On optical media storage devices, each of the sectors consists of two major parts; an Identification field (ID) used by the device controller to locate a particular sector by a physical address and a data field for storing data. The informational content of the ID's on hard sectored optical disks are indelibly recorded, as by a stamping/molding process, on the medium at the time of manufacture. Other data storage formats also are usable to practice the present invention, such as the known count-key-data (CKD) and extended count-key-data (ECKD) formats used on many magnetic disk media.
An FBA device attached to a host via the known Small Computer Standard Interface (SCSI) must provide the capability to resolve a Logical Block Address (LBA) used by SCSI architected direct-access data storage devices to address fixed sized units of storage to a unique physical address (track and sector) on the medium. The SCSI attached FBA device provides to the host a contiguous address space of N (N is a positive integer) storage locations which can be accessed for reading or writing in any sequence. Each LBA directory structure (addresses ranging from 0 to N) is the addressing mechanism used to store and retrieve data blocks in the SCSI-FBA environment (some FBA devices also provide the capability to address the storage space using the physical address).
As can be seen from the preceding paragraphs, the principal problem facing a designer of a storage system using data compression techniques in the SCSI-FBA environment is to provide a mechanism by which fixed size units of data, herein termed data blocks, in an input data stream can be recorded in a variable amount of medium storage space and still maintain addressability to the unoccupied storage space and provide for addressability to the recorded data blocks.
Since many optical disks today are of the removable type, it is further desired to enable the removable data storage medium to be self-describing as to compressed and uncompressed data.